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ABSTRACT 

In this study, relationships were examined among 
three aspects of the implementation of complex instruction, an 
innovative approach to teaching in heterogeneous classrooms. These 
three aspects were: (1) teachers* use of the complex instruction 
strategies in the classroom; (2) systematic and comprehensive 
feedback received by teachers from the developers of the innovation; 
and (3) teachers' conceptual understanding of the theoretical and 
empirical knowledge base underlying complex instruction. For this 
study, data collected as part of a larger project conducted by the 
Stanford University School of Education were used. Thirteen teachers 
in 5 elementary schools constituted the sample; data sources were a 
minimum of 20 classroom observations of each teacher, questionnaires, 
structured and open-ended interviews, and records of collaborative 
meetings between teachers and supportive supervisors. The findings 
indicated that the frequency of feedback sessions was positively and 
significantly related to the quality of the implementation of the 
innovation, and the teacher's conceptual understanding of the 
knowledge base. Teachers' conceptual understanding was positively and 
significantly related to the more sophisticated teaching strategies 
but showed no relationship to the more routine behaviors. Case 
studies illustrate the quantitative findings. Theoretical and 
practical implications of the findings are discussed. (Author/ AMH) 
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Abstract 

In this study, we examined the relationships between three aspects of the 
implementation of Complex Instruction, an innovative approach to teaching in 
heterogeneous classrooms. These three aspects were: 1) teachers' use of the 
strategies in the classroom, 2) systematic and comprehensive feedback received by 
teachers from developers of the innovation, and 3) teachers* conceptual 
understanding of the theoretical and empirical knowledge base underlying Complex 
Instruction. 

We found that the frequency of feedback sessions was positively and 
significantly related to 1) the quality of the implementation of the innovation, and 
2) to the teacher's conceptual understanding of the knowledge base. Teachers' 
conceptual understanding was positively and significantly related to the most 
sophisticated teaching strategies but showed no relationship to the more routine 
behaviors. We also used case studies to illustrate our quantitative findings. 

Theoretical and practical implications of our findings are discussed. 
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Looking in Mirrors: Teachers Doing, Seeing, Knowing 

How do teachers make changes during the implementation of an instructional 
mnovation? The following comments are from teachers during their first year of 
implementation of complex instruction, an innovative instructional program 
developed at Stanford University: 

First teacher: All the pieces of the puzzle don't make sense at first. But 
afterwards when you go out and implement it, it makes sense. 

Second teacher: Feedback is important so you know whether you are doing 
what the program is asking you to do and whether children are reaping the 
benefit. And (you are reminded of) some things you could do to better 
implement it. 

Third teacher: Now (at the end of the first year of implementation) my 
students have gotten so accustomed to helping one another, they have trouble 
not doing it. 

Purpose 

Do teachers adopt new instructional approaches automatically on command, 
do they copy models, or do they translate and adapt theory to fit their own needs? 
Some developers of innovations attempt to produce routine "teacher-proof" 
materials and textbooks that dictate teachers' actions and bypass their decision- 
making and thought processes. Other innovators believe that teachers make sense 
oiit of an incredible diversity of information sources, including knowledge about 
the theoretical bases and instructional processes recommended for implementation 
of the iimovation, theoretical and empirical literature relevant to the innovation, 
information about individual students, and teachers' self knowledge about their own 
interactions with students (Shulman, 1974). 

In this study we examine relationships between three aspects of the 
implementation of a complex instructional innovation: (1) teachers' use of the 
strategies in the classroom, (2) systematic and comprehensive feedback received 
by teachers from developers of the innovation, and (3) teachers' conceptual 
understanding of the theoretical and empirical knowledge base underlying the 
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innovation. In part, we concur with Joyce & Clift (1984), that: 

... helping teachers use any complex pattern of behavior requires instruction 
in the theory of the behavior, modeling, opportunities to practice the 
behavior, and extensive coaching as teachers work with the behavior in their 
own classrooms. Teachers need a supportive environment in which they are 
free to make mistakes and receive ample, non-threatening opportunities for 
corrective feedback. 

Joyce and Clift offer a framework synthesized from information, observation 
about the current states of schools and teachers, and value positions. They argue 
that a major task in any reform effort is to generate testable approaches to the 
substance and process of teacher education and to wed reform with research. The 
present study is a response to their summons and others that are similar. We 
extend Joyce and Clift*s notion of the "coaching" of teachers as the attempt is made 
to change their teaching strategies. We make the prediction that teachers who 
clearly understand the conceptual underpinnings of a complex innovation will learn 
to implement that innovation better than teachers who do not have a firm grasp of 
the theories. We collected data to test our prediction. 

The present study makes three main contributions to research on the 
implementation of educational reforms. The first contribution is the use of 
comprehensive, systematic feedback to teachers on their classroom implementation 
(Dombusch & Scott, 1975). Especially important was our insistence on feedback 
that was soundly based. Soundly based feedback goes beyond behavioristic models 
of coaching that focus on the teacher's behavior without relating behavior to the 
theoretical foundations of the innovation. Moreover, soundly based feedback is 
grounded on systematic observations of teachers' practice in classrooms. The 
specificity of feedback resembles a mirror held up to teachers to view their own 
practice. Indeed, for selected teachers, the regular feedback was enhanced by 
video tapes of teachers (Benton, in progress).* 



^Benton, J. (forthcoming). Treating status problems in the 
classroom: Training teachers to assign competence to students 
exhibiting low status behavior in the classroom. Stanford 
University, Stanford, CA: Dissertation in progress. 

Supervisors assisted teachers in the analysis of video tapes. They 
helped teachers focus on specific events that wsre difficult for 
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The criteria for feedback and standards for implementation were clear in the 
present study. Teachers knew which behaviors encouraged children to talk and 
work together, which in turn are the behaviors that produce students* learning. 
Standards for implementation were based on previous research that documented the 
relationships between the essential features of the program and student 
achievement. 

Feedback was delivered by program developers in non-evaluative, 
collaborative contexts. Teachers* performance was neither a basis for merit pay 
nor any other organizational reward. Ellis (1987) conceptualized this kind of 
feedback by program developers as supportive supervision; feedback given at 
regular intervals and well structured to focus on teachers' performance of non- 
routine teaching behaviors that were designed to increase students* learning gains. 
For each feedback session, supportive supervisors used graphs based on at least 
three classroom observations. On the graphs teachers saw visual representations 
of the proportion of their desirable to undesirable teaching behaviors. 

A second contribution of the present study is the focus on teachers* 
knowledge of how to implement the innovative program, as well as why . We 
argue that teachers are professional decision-makers who reflect on the theoretical 
as well as the empirical features of an innovation. 

A third contribution of this study is the examination and monitoring of 
implementation over time, repeatedly, in the same classrooms, taking into account 
the possibility that teachers may be cognizant of the theoretical foundations of 



teachers to respond to in the ongoing life of the classroom. For 
example, when low status students displayed competence for a 
variety of abilities, teachers tried to make those abilities public 
in the classroom by making specific remarks about them. Video 
tapes of students who displayed special abilities offered teachers 
opportunities to practice appropriate responses and gave them 
models for identifying such opportunities. Without such practice 
and specific analysis, teachers tended to stereotype low status 
students as not committed or lazy. It was difficult for teachers 
to identify students' multiple abilities related to the learning 
task at hand and offer public and specific praise. The innovative 
program required teachers to analyze events quickly, understand 
what the task called for, and make known what the student could do. 
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teaching practice in addition to their focus on classroom action. This perspective 
departs from that of Shavelson (1988), who suggests that teachers may focus 
mainly on classroom action and that research benefits the practice of teaching only 
when it improves practical arguments in the minds of teachers. Indeed, many 
programs are not based on a theoretical framework or empirical data. However, 
when a theoretical and empirical knowledge base exists, we expect a different 
situation. Lotan (1985) found significant positive links between teachers' 
conceptual understanding and quality of implementation in her empirical study of 
a complex innovation that contained a clear theoretical knowledge base supported 
by empirical evidence. 

Theoretical Framework 

Organizational theorists tell us that the more complex the worker's task 
becomes, the more advantageous it will be to use lateral, rather than top-down, 
channels of communication (Perrow, 1967). The use of lateral communication 
channels helps workers by increasing opportunities for them to use each other as 
information resources. When teachers use a complex instructional technology 
involving many non-routine features, they experience higher levels of uncertainty 
than they would within more routine technologies. Organizational sociologists 
theorize that lateral communication affords workers opportunities to obtain and 
process information and to gain assistance, thereby lowering their uncertainty and 
increasing their productivity (Galbraith, 1973). If the theory holds for teachers 
using a complex instructional technology, then those teachers who talk about their 
teaching will improve their classroom performance as they gain more knowledge 
and eliminate uncertainty by collaborating. 

March and Simon (1958) argue that in less routine task environments, 
individuals engage in wider and higher-level search procedures than in more 
routine task environments. Wider search procedures are undertaken when 
individuals search for mformation from those in lateral positions, rather than above 
them in organizational hierarchies. Higher-level searches occur when individuals 
ask experts for information. In the present study, where the task environment of 
teachers was largely non-routine, supportive supervisors provided an important 
arena for collaboration in the feedback sessions with teachers. Supportive 
supervisors observed teachers and gave them specific, systematic feedback on their 
implementation of complex instruction. The supportive supervisors wer^ not linked 
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with administrative evaluators in the school organizational hierarchies. However, 
they were experts on the implementation of complex instruction, and they were 
able to provide teachers with crucial information. Teachers were able to inquire 
freely and exchange information with supportive supervisors. 

Although previous researchers have examined lateral communication 
(Rosenholz, 1989; Littie, 1982), few have examined what is being discussed. In 
the present study the content of feedback sessions was known to be specific to 
implementation of the innovation because topics were closely monitored. Tlie 
feedback sessions were designed so as to be closely aligned with the instructional 
technology. 

Ellis (1987, 1990) found that, under certain conditions, this collaboration was 
associated with teachers' use of non-routine teaching strategies (and non-use of 
routine strategies) when they used a complex instructional technology designed to 
encourage students to talk and work together. The first hypothesis is: 

Given a complex innovation, supportive supervision and quality of 
implementation will be positively related. 

The remaining question for this study is; What is the relationship of 
teachers' collaboration (with the supportive supervisor) to teachers' understanding 
and implementation? Following Scott (1981), Lotan (1985) argues that the facets 
of classroom technology are materials, operations, and knowledge. Practitioners 
use two kinds of knowledge: (1) highly codified, regulated, and explicit knowledge 
and (2) complex, abstract knowledge typically used for problem solving. Although 
abstract knowledge is not situation-specific, this body of knowledge is organized 
and structured so as to permit systematic application of its concepts and principles. 
Lotan (1985) discovered that teachers' conceptual understanding of a complex 
instructional technology was positively associated with their implementation of non- 
routine teaching strategies in a complex instructional technology. Replicating 
Lotan's study in this different sample of teachers, we hypothesize that: 

Given a complex innovation, teachers' understanding of the underlying body 
of knowledge will be positively related to teachers' classroom performance. 

Understanding develops from three different experiences: (1) the workshop, 
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(2) a practicum experience during the workshop and implementation in the 
classroom, and (3) sound feedback that makes connections between theory and 
implementation. Thus, the third hypothesis is; 

Given a complex innovation, feedback by supportive supervisors will be 
positively related to teachers' conceptual understanding of the theoretical 
knowledge base. 

The theoretical model is neither linear nor causal, but interactive. 
Therefore, we will examine the various relationships one by one rather than in a 
simultaneous linear mode. Figure 1 . is a visual representation of the model. 

Figure 1 about here 



Methodology 

Setting and Sample 

For purposes of this study, we used data collected as part of a larger project 
conducted under auspices of the Program for Complex Instruction at the Stanford 
University School of Education. Thirteen teachers in five elementary schools 
constituted the sample. The complex instructional approach was used in 
conjunction with Finding Out/Descubrimiento, an English/Spanish math and science 
curriculum for elementary schools. (For a detailed description see Cohen, 1986, 
and Cohen and Lotan, 1989). Data sources were a minimum of 20 classroom 
observations of each teacher, teacher questionnaires, structured and open-ended 
interviews, and records of collaborative meetings between teachers and supportive 
supervisors (Stanford staff). Final interviews were designed to reveal teachers' 
conceptual clarity and understanding of the theory, as well as the basic principles 
and features of the program. 

The classrooms in this study had the following characteristics (adapted from 
Lotan, 1990): 

1. students were from racially, ethnically, linguistically, and socially mixed 
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backgrounds, 

2. materials (which include manipulatives) and activities used by these 

students were diverse and differentiated, 

3. task activities were such that multiple intellectual abilities were needed 

for successful completion, providing opportunities for all students to 
contribute and to serve as intellectual resources, 

4. to stimulate peer interaction, tasks were uncertain from the students' 

point of view, open-ended, and intrinsically interesting, 

5. to maximize student participation and peer interaction, instruction was 

conducted in heterogeneous small groups, with emphasis on group 
norms and student roles as well as individual accountability, 

6. groups were differentiated, with each group working on a different 

activity with different materials, 

7. teachers implemented an instructional approach that emphasizes non- 

routine teaching behaviors such as delegating authority to students, 
thus making them responsible for their own and their group mates' 
task engagement and learning; 

8. to equalize and balance interaction in the heterogeneous groups, teachers 

used strategies designed to counteract unwanted domination of the 
group by its high-status members; 

9. complex instruction, the classroom technology, is grounded in a well 

defined and clearly stated, theoretical and empirical body of 
professional knowledge, 

10. teachers participated in extensive and thorough theoretical as 

well as practical training and received feedback on their performances 
in the classroom during the school year. 

In Complex Instruction, the lesson is usually divided into three parts: 1) a 
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brief orientation to the activities by the teacher, 2) the learning center timft during 
which students work in small groups ont he day's tasks, and 3) a wrap-up during 
which reporters from each group report to the teacher and to the whole class about 
the result of their work in the groups. Given the characteristics of the tasks and 
the organization of the classroom, during learning center time students engage in 
exciting scientific activities. They read and discuss the instructions to the tasks, 
they hypothesize about reasons and outcomes, they perform experiments, they 
observe, measure, calculate, and record their findings (Lotan, 1990). 

Procedures 

The first hypothesis of the study states that there is a positive relationship 
between the amount of supportive supervision and tlie quality of the teacher's 
classroom performance as prescribed by the program developers. Supportive 
supervision consists of a) Systematic observations of the implementation of the 
program in the classroom and b) an in-depth, problem-solving analysis of these 
observational data during structured feedback meeting between the teacher and a 
supportive supervisor. In the present study, such supportive supervision was 
provided to the teachers by staff members of the Stanford project. Selected 
teachers volunteered to receive additional feedback and training by analyzing 
videos of their students' and their own performances in the classroom. 

The amount of supportive supervision is indicated by the frequency of 
feedback meetings between the teacher and a Stanford trainer/staff developer. 
Although the model of staff development for complex instruction prescribes at least 
three such meetings during the academic year, at the time of data collection for 
this study (the 1984-85 academic year), there was considerable variation in the 
number of these meetings among the teachers in the sample. This variation came 
about for a number of reasons: a) Stanford was conducting an experiment in which 
four out of thirteen teachers received video feedback focusing on specific non- 
routine behaviors (status treatments) in addition to the regular feedback; b) at first, 
many teachers felt threatened by having classroom data presented to them and 
perceived the feedback as an evaluation. As a result, some teachers successfully 
avoided setting up meetings with the staff, and sometimes even cancelled meetings 
already planned; c) serious shortages of staff members on the Stanford project 
made them less persistent and consistent in pursuing the teachers and bugging them 
to set up meetings. Thus, one teacher received no feedback before the end of the 
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observations, six teachers participated in two sessions, three teachers in three and 
one teacher participated in four feedback meetings. These numbers include regular 
as well as video feedbacks. Measures of frequency of supportive supervision were 
taken from the records kept by the Stanford staff and were substantiated in 
debriefing interviews of the teachers. 

We categorized teachers' classroom performances in two ways according to 
the prescriptions of the program and as derived from the theoretical and empirical 
knowledge base: the non-routine behaviors and those behaviors that indicate direct 
supervision by the teacher. In simple terms, the former are the desirable behaviors 
and the latter are the behaviors to be minimized, particularly at learning center 
time. Given the basic differences in the context of instruction during orientation 
and wrap/up and at learning center time, i.e. whole class vs. small group 
instruction respectively, date were collected separately for these two kinds of 
situation. 

Non-routine behaviors are those teaching acts in which the tt.'clier engages 
when she or he has the option to choose from a repertoire of essentially varied 
behaviors. Non-routine behaviors in teaching are, therefore, the result of a 
decision-making process that Shavelson (1973) has called the basic teaching skill. 
In complex instruction, the common and familiar repertoire of teacher behavior has 
been expanded: new strategies are introduced to the teacher who implements the 
program in the specific setting of small group instruction when the overall 
organization and social system of the classroom is changed from the traditional, 
whole-class setting. 

Based upon theoretical formulations and empirical findings, the program 
developers at Stanford have defined the criteria for superior classroom performance 
during implementation of complex instruction. For example, a crucial feature of 
complex instruction is treatment of status problems in the small groups. When the 
teachers talk about multiple abilities and assign competence to low status students, 
they reduce the severity of status problems in their classrooms, thereby equalizing 
rates of participation of low and high status group members. 

In the context of complex instruction, non-routine teaching behaviors include 
giving specific feedback to individuals and groups, stimulating and extending 
students' thinking, and treating status problems. They are non-routine because no 
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specific program (as defined by March and Simon, 1958) or pre-planned response 
can be provided to the teachers. Given the fluidity of the situation and the 
considerable repertoire of potentially appropriate responses, teachers have to search 
their minds and make decisions as to which behavior to use, given particular 
situations. For example, after observing the operations of the students at a 
particular learning center, the teacher might decide to extend learning by providing 
additional examples; he or she might decide to address the group as a whole or to 
provide feedback to an individual. Finally, the teacher might decide to move on 
to the next center without saying anything at all. Thus, teachers' use of non- 
routine teaching behaviors was measured by the following indicatoi^, all of which 
are positive: 1) rate of teacher's talk about multiple student abilities; 2) rate of 
giving specific feedback to students; 

3) rate of teachers's talk about extending activities; 4) rate of teacher's talk about 
student thinking. 

In contrast to the non-routine behaviors explained above, some teaching 
behaviors are of more routine nature and they are also an essential part of 
conducting the classroom effectively. However, these routine behaviors need to 
be minimized in complex instruction. Research has shown that student interaction 
(the main predictor of learning) is short-circuited when teachers help student with 
the task, when they provide too much information, or when they question too often 
(Cohen, Lotan with Leechor, 1989). Thus, teachers' use of routine teaching 
behaviors was measured by the following indicators of implementation, all of which 
are negative: 1) rate of teacher's facilitating students' work, solving problems for 
students that could be solved by the students themselves; 2) rate of teacher's 
disciplining students; 3) rate of teacher's questioning students when the 
questioning is accompanied by high rates of facilitating. 

Data about teachers' classroom performance were obtained from systematic 
observations of the teachers during implementation of complex instruction. 
Observers used a standardized teacher observation instrument to categorize and 
tally teachers' verbal interactions during ten-minute intervals. Inter-observer 
reliability was established at over 90% agreement with a criterion observer. 

The teacher observation instrument was used to guide observations of 
implementation for each of the 13 teachers approximately 20 times during the 
school year. During an average classroom period of about 45 minutes, each 
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observation lasted 10 minutes while students were at learning centers and 10 
minutes while the teacher was conducting an orientation or a wrap-up. 

The first hypothesis of the study is translated into the following specific 
predictions: 

1) The frequency of feedback meetings between teacher and supportive 
supervisor will be positively related to the average rate of non-routine 
teaching behaviors during a) learning center time and, b) during orientation 
and wrap-up. 

2) The frequency of feedback meetings between teacher and supportive 
supervisor will be negatively related to the average rate of routine teaching 
behaviors during a) learning center time and, b) during orientation and wrap- 
up. 

The second hypothesis of the study focuses on the relationship between 
teachers' understanding of the underlying knowledge base to complex instruction 
and her classroom performance. We hypothesized that when the teacher has a 
better grasp of the theoretical framework and the empirical findings, she will 
minimize the routine behaviors and will maximize the non-routine behaviors. 
Methodologically, the measurement of teachers* conceptual understanding of this 
knowledge base presented a serious challenge. We clearly could not administer a 
test to them. However, based upon a previous study (Lotan 1985), we were able 
to infer conceptual understanding from teachers* answers to certain questions in an 
open-ended interview. For example, we asked teachers the following questions: 
"Please think of the child in your classroom who is seen by his classmates as slow 
and who does not have many friends. How did this child function during FO/D? 
Do you think the program made him feel more competent? What in the program 
do you think could be responsible for helping a child like that? Did you make any 
special effort to help this child during FO/D? 

The interview, administered to the teachers at the end of the academic year, 
had two overt and one covert purpose: it was designed so that teachers* conceptual 
understanding could be inferred from a) teachers* responses to questions that 
asked for evaluation of their training and from b) their responses to situations that 
presented them as potential disseminators of information about complex instruction. 
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The context of the verbal information expressed by the teachers during the 
interview was evaluated, scored and quantified by comparing it to standards 
derived from the theoretical formulation by the developers. Conceptual 
understanding of the following aspect of complex instruction were evaluated: 1) 
goals and scope of the program; 2)principles of cognitive learning; 3) classroom 
management in small-group instruction; 4) treatment of status problems; 5) the 
importance of organizational support for complex instruction. (Guidelines and 
additional details for scoring are described in Ix)tan, 1985). 

Two independent scorers interpreted and coded the content of the interviews. 
Inter-scorer reliability was 86% agreement. The scorers were careful to 
differentiate between a) verbal information and cues from which conceptual 
understanding of an item (key concept or principle) could be inferred and b) 
reports about behavioral manifestations of a particular item. After coding the 
interview, the scorer arrived at a count of the total number of times a particular 
item was mentioned as well as the total number of items to which references were 
made in the interview. For the measure of overall teacher understanding, we 
simply added the total number of items coded in the interview. 

The second hypothesis is translated into the following predictions: 

3) Teachers* overall understanding of the knowledge base of complex 
instruction will be positively related to the rate of non-routine teaching 
behaviors a) during learning centers, and b) at orientation and wrap-up. 

4) Teachers* overall understanding of the knowledge base of complex 
instruction v/ill be negatively related to the rate of routine teaching behaviors 
a) during learning centers, and b) at orientation and wrap-up. 

The third hypothesis deals with the relationship between the frequency of 
feedback meetings and the teachers* overall conceptual understanding. The 
following prediction is derived from the third hypothesis. 

5) The frequency of feedback meetings between the teacher and the 
supportive supervisors will be positively related to the teacher*s overall 
understanding of the program. 
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Results 

Table 1 shows the descriptive statistics of the variables in the study. The 
mean value of the variable of teachers* overall conceptual understanding was 94.82 
and the standard deviation for that variable was 26.68. There was a considerable 
range in this variable as mdicated by the minimum and maximum values. The 
statistics of the next variable, frequency of feedbacks, (x= 1 .77; sd= 1 . 17) reflects 
the variation in the number of feedback meetings for the teachers in the sample. 

Table 1 about here 

Table 1 also shows the average rate per 10 minutes of observation period of 
selected teaching behaviors. Among these behaviors, the highest average rate was 
for direct supervision at learning centers (x=21.2, sd= 12.46). The lowest 
average rate was for non-routine behaviors during orientation and wrap-up 
(x=6.49; sd=3.51). In general, the average rates of non-routine behaviors were 
lower than those classified as supervisory. 

Table 2 about here 

Table 2 shows the strength of relationships among variables of the study. 
All the relationships between the frequency of feedback and the teaching behaviors 
were in the direction predicted and statistically significant. Frequency of feedback 
and direct supervision at learning centers and during orientation and wrap-up were 
both significantly negatively related: r=-0.53, p<.05 and r=-0.65, p<.01 
respectively. Frequency of feedback and non-routine behaviors at learning center 
and during orientation and wrap-up were both significantly positively related: 
r=.57, p<.05 and r=.68, p<.01 respectively. 

The relationship between teachers' overall understanding and teaching 
behaviors were in the predicted direction. However, only the relationship between 
overall understanding and non-routine behaviors at learning centers achieved 
statistical significance, r = .55, p < .05. 

There was also a statistically significant positive relationship between the 
frequency of feedbacks and teachers' overall understanding (r=.62, p<.05), 
meaning that those teachers who participated in more feedback meetings also had 
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a better understanding of the underlying knowledge base of the program. 

Discussion 

The data were supportive of the predictions that had to do with the 
relationships between feedback and the teachers' classroom performance, on the 
one hand, and between feedback and the teachers' conceptual understanding on the 
other. The results of data analysis relating to predictions about the relationship 
between teachers' conceptual understanding and their classroom performance were 
mixed. 

When classroom technology is complex, it is uncertain and requires constant 
non-routine problem solving by the teacher. Supportive supervision then becomes 
necessary to promote and maintain quality imple-nentation. Both the technology 
and the content of the supportive supervision described in this study were quite 
different from the average instructional innovation or the "coaching" that seem to 
be wide-spread nowadays. As presented in this study, the feedback to the teachers 
was highly structured, task-oriente»' and performance-specific. The innovation was 
complex and the teachers' behaviors highly non-routine according to the dimensions 
defined by the program. It could well be the case that other, less complex and less 
deep-reaching educational iimovations do not need such an elaborate support system 
in order to be well implemented and in order to survive. 

Teachers' conceptual understanding of the underlying body of knowledge 
seems particularly relevant to the implementation of non-routine behaviors during 
learning center time. It is at this time that decision-making is most challenging 
because the teacher needs to "think on her feet. " On the contrary, orientations and 
wrap-ups are more easily planned and thought out ahead of time. We speculate 
that more routine strategies, which can be planned ahead for orientations and wrap- 
ups, can be employed by teachers without so great an overall understanding of the 
theoretical bases of the program. 

Of particular importance is the existence of a well-defined and immediately 
relevant body of knowledge that underlies this complex technology. Unlike many, 
if not most educational innovations, complex instruction has a solid theoretical 
basis that is supported by educational research. It includes definitions of social 
science concepts as well as predictions and explanations of relationships among 
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these concepts in classroom settings. The body of knowledge posits general 
pedagogical rules and principles and the conditions for their applications. 

Examples from the case studies. 

Next, we return to the teachers' interviews to find out what they said about 
the feedback they received from supportive supervisors. Mrs. Putney best 
exemplifies the predicted results. She was observed and received feedback in four 
formal sessions. Her implementation in the classroom was exemplary. She had 
the highest ranking scores on non-routine teaching behaviors and overall 
understanding of the program. She ranked lowest and second lowest, respectively, 
on routine teaching behaviors at orientations and wrap-ups. She said: 

I wouldn't have minded if the feedback sessions had been more frequent and 
earlier in the year. They were extremely useful because they were so 
specific. I got a very clear picture of what was going on. I now have a 
much more clear idea of what it is that I said at the beginning of the year 
and how I've changed. In the beginning a comment like "Rosa has done 
thus, thus, thus..." would never have struck me as a comment that would fit 
anywhere. I would never have recognized it as specific feedback. I can 
see the same principles apply with us (teachers) and the children. That is, 
the more specific feedback you get, the better you understand. 

Mrs. Putney saw that the process of her own learning to implement the 
program was analogous to processes of students' learning. The same principles 
applied. The more specific she was in her comments to individual children, the 
more they learned. In turn, the more specific the feedback she received from her 
supportive supervisor, the more she learned about teaching in the group work 
setting. She gave evidence that she was able to view her own learning objectively. 

Consistent with the predictions of the present study but at the other end of 
the scale, Mrs. Valenza's story was not a tale of success. She participated in only 
one feedback session, her overall understanding of the program was low, and her 
performance of routine teaching behaviors at learning centers was higher than any 
other teacher's. For routine teaching behaviors at learning centers, Mrs. Valenza's 
mean score was 44.3; the sample mean was 21.2. At the end of the year her 
comments on the feedback were as follows: 
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Jane (Mrs. Valenza's supportive supervisor) said I was supervising kids a 
lot. After our first (feedback) session I tried not to do that a lot. It was 
very helpful. Jane understood that one student needed more help from the 
teacher than the other students. The time I spent directly supervising them 
was well spent time. Once Jane showed me the percentage graphs (of my 
teaching behaviors), she understood why I had to supervise so much. 
Especially since I have a second grade. After Jane understood the 
conditions, I don't think there were any problems. I know she wanted me 
to do more. She suggested a couple of things, like: let the aide do more 
questioning, stand back nd watch kids in their roles, give specific feedback. 
I think I'll use those suggestions next year. 

Mrs. Valenza directly resisted acting on the feedback based on graphs of her 
teaching behaviors. Jane tried to show her that she was giving students too much 
help and supervising too directly. Mrs. Valenza had ready excuses for refusing 
to take suggestions and ingratiated herself by resolving to Jane's suggestions next 
year. Blaming the grade level she taught and a student who needed extra help, she 
failed to recognize principles that applied to her own learning processes. 
Observers in Mrs. Valenza's classroom noted that she frequently apologized 
because she her second graders didn't "get" the science concepts they explored, 
even when she gave explicit help and instruction. She didn't seem to understand 
that one of the goals of the program was to encourage children to ask questions, 
observe, and discover independently. 

Mrs. Putney and Mrs. Valenza were exemplars of the predictions made in 
the present study and the findings. Feedback and implementation were strongly 
associated. Overall understanding of the program was closely associated with 
feedback and non-routine teaching strategies at learning centers. At orientation and 
wrap-up, non-routine strategies and overall understanding were associated, but the 
correlation did not reach a level of significance at p<.05. 

Next we will describe Miss Escher and Mrs. Rickover, two teachers who 
resembled the results of the present study more closely than Mrs. Putney and Mrs. 
Valenza. Miss Escher participated in three feedback sessions, two of the sessions 
based on videotapes of her teaching. Classroom observations disclosed her as a 
highly skilled teacher who used many of the program's recommendations 
frequently. She ranked second highest in the use of non-routine teaching 
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behaviors. Surprisingly, she had the lowest score on overall understanding of the 
program. Her comments on the feedback seem to reveal an emphasis on negative 
information. For example, she said: 

Many times I got caught up with facilitating (helping too much). It was 
helpful for (a colleague) to observe me to remind me to stay in my role. I 
think the purpose of the feedback is to help us to be aware of our mistakes 
and help us improve on them and to help us know what we're doing right. 
What was useful? Pointing out the positive things and negative things. 
Seeing things I wasn't even aware of. Everybody hates to know they're 
doing something wrong or negative. In the long run it was good. Once in 
a while when I had a good day, no one was there. 

Miss Escher complained that Felicity, her classroom aide, liked to "hover" 
over students while they worked. She said she had "tried to talk to Felicity, but 
she's established in her ways." Although Miss Escher was self-critical and 
mentioned habits she tried to eliminate, such as trying not to facilitate, hover, or 
help students too much; her observed classroom behavior was quite positive. She 
asked children stimulating questions, extended activities, talked to children about 
multiple abilities and thinking processes. In the context of trying to model teaching 
behaviors for her aide to adopt, she may have been able to "step outside" her own 
perspective and see herself objectively. Although she had a poor understanding of 
the program itself, the feedback sessions may have given her enough information 
and visual imagery to enable her to act in a recommended way without being well 
grounded in the theory. 

Ms. Rickover participated in only one feedback session. Following the 
original predictions in the present study, she had high scores on routine teaching 
behaviors, especially at learning centers. Contrary to predictions in this study, her 
score on overall understanding of the innovative program was fourth from the 
highest. She said she "felt frustrated because there was always something 1 didn't 
do." Like Miss Escher, Ms. Rickover expressed some dismay over conflicts with 
her classroom aide and a lack of time for communication. She said: 

I never had any planning time with my aide. She had a hard time knowing 
when to discipline and how to discipline. And so she didn't know when it 
was hovering and when it was disciplining. She had a hard time. Toward 
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the end (of the year) I would lose my patience a little bit and I felt bad 
because we (the aide and I) never got to stand and talk, but I just told her - 
look, you handle it. Don't come over and ask me. You can handle it. 

On the subject of feedback, Ms. Rickover said it was useful because it gave 
her positive reinforcement and because it made her aware of opportunities she had 
missed to talk about such things as multiple abilities. Although Ms. Rickover 
understood the program well, her failure in classroom implementation was 
compounded by her failure to model the recommended teaching behaviors for her 
assistant. She had not succeeded in "stepping out of herself to look at herself as 
a teacher, as an instructor for her aide, or as a teacher-learner. Instead, she was 
caught in feelings of frustration, complaining that there was "always something I 
didn't do." Ms. Rickover did excessive amounts of questioning and "hovering." 

What can we learn from the individual cases? Considered as a pair who 
were alike in successful implementation and receiving greater amounts of feedback, 
Mrs. Putney and Miss Escher were both enthusiastic. Their interviews had a tone 
of jumping on the program bandwagon. Mrs. Putney was a teacher-leader, 
observing other teachers and giving them feedback, working with her aide to 
develop teaching skills, and writing grant proposals to pay for science lab 
materials. Miss Escher was a follower. She was attentive to the behavior of 
another teacher she admired and whose implementation of the program was 
excellent. Like Mrs. Putney, Miss Escher was enthusiastic, but unlike Mrs. 
Putney, perhaps she relied more on observation and copying models than on 
understanding of the program for successful implementation. 

Mrs. Putney looked at her own learning process objectively, as a teacher 
learning a new program. She understood that seme of the principles of learning 
that apply to young children might also apply to teachers. She talked about specific 
feedback helping children, as well as teachers. Understanding the theoretical basis 
for the program helped her apply her knowledge to practice. For Miss Escher, 
application was a matter of mirroring images. Objective knowledge of herself as 
a learner was not necessary for her success in implementation of the innovation. 

What, then, can we make of Ms. Rickover 's high overall understanding and 
failure in implementation? She did not attempt to model recommended behaviors 
for her assistant, she was not a "teacher leader" or and enthusiastic bandwagon- 
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rider, and she did not have an objective view of her own learning processes. What 
was the source of her high level of program understanding? We can only surmise 
that she had ample opportunities to learn about the program, as did all the teachers 
in the study, and that she gleaned the information froii sources other than 
collaborative sessions with colleagues. A footnote on Ms. Rickover's story is that 
she had extremely stressful problems in her personal life during the entire school 
year. We have no way of knowing how much those problems influenced her 
classroom performance. 

Mrs. Valenza's low scores on all measures seem to be self explanatory, but 
what prevented her from gaining knowledge about the program, like Ms. Rickover? 
She was convinced that her students needed extra help, in spite of the program's 
recommendations. Was there a cultural gap, some key misunderstanding, a refusal 
to jump on the program bandwagon, or an inability to see herself as a teacher from 
an objective point of view? 

Implications 

Implications for researchers . 

Conceptual understanding of a complex instructional program seems to be 
necessary for the most sophisticated aspects of implementation: non-routine 
teaching behaviors during learning center time. Conceptual understanding seems 
to be somewhat less necessary for less sophisticated aspects of the implementation: 
performing non-routine behaviors during orientation and wrap-ups and refraining 
from routine behaviors during learning centers and orientations and wrap-ups. An 
interesting direction for researchers would be comparisons of conceptual 
understanding required by teachers for implementation of more complex 
instructional programs and understanding required for implementation of less 
complex programs. 

Feedback (supportive supervision) appears to be an important condition for 
successful implementation of all aspects of complex instruction. Surprisingly, 
feedback does not necessarily seem to improve implementation by increasing 
knowledge about the program. Feedback must have other beneficial effects. 
Perhaps there is a social effect: teachers may learn to take positive attitudes 
toward the new program, they may learn to reflect on their own learning 
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processes, and they may learn from concrete cases when colleagues give them 
specific feedback. These possibilities should be examined. 

Implication s for practitioners. 

Organizational supfjort for collaboration and feedback is essential for the 
successful unplementation of sophisticated instructional programs involving non- 
routine teaching strategies. However, school boards and administrators will find 
that the expense of providing feedback, meeting teachers' needs to understand the 
conceptual foundations of complex instructional programs, is exorbitant and 
impractical in today's educational world. Complex instructional programs 
attempted without sufficient organizational support will have a high failure rate, 
even when they are hailed as valuable educational improvements. 
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Table 1 

Descriptive Statistics of Variables of the Study 

VARIABLE 

NAME MEAN SD RANGE (N) 



Over all conceptual 

understanding 94.82 26.68 49 - 161 
Frequency of 

Feedbacks 1.77 1.17 0 - 4 (13) 
Direct supervision 

learning centers 21.20 12.46 10.42-44.27 (13) 
Direct supervision 

orientation/wrap-up 12.55 5.61 6.43 - 25.43 (13) 

Non-routine 
behaviors 

learning centers 7.17 2.68 3.5 - 12.25 (13) 

Non-routine 

behaviors 

orientation/ 

wrap-up 6.49 3.51 3.09 - 13.29 (13) 
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Correlation Matrix of Variables of the Study 
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1 Overall 
conceptual 
understanding 

2 Frequency of 
feedbacks 

3 Direct 
supervision 
learning 
centers 



1.00 



.62* 
(11) 



-.36 
(11) 



1.00 



-.53* 
(13) 



1.00 



4 Direct 
supervision 
orientation/ 
wrap-up 



-.20 
(11) 



-.65** 
(13) 



.40 1.00 
(13) 



5 Non-routine 
behaviors 
learning 
centers 



6 Non-routine 
behaviors 
orientation/ 
wrap-up 



.55* 
(11) 



.43 
(11) 



.57* 
(13) 



.68** 
(13) 



-.17 
(13) 



-.28 
(13) 



-.52 1.00 
(13) 



-.35 
(13) 



.47 

(13) 



1.00 
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